Approximating Sequences of LPs for Networks and Related MDPs

A standard method of solving queueing network control MDPs with uncontrolled arrivals is to truncate the state space by turning o¤ arrivals at some queue length. However, this method requires large queue lengths to be accurate. This paper formulates approximate linear programs (ALPs) that give lower bounds average cost. Any di¤erential cost is allowed below some queue length but it is approximated by a linear form beyond that point. For reentrant lines and a certain approximation, we show that the ALP converges to the average cost as this queue length threshold becomes large. The result is analogous to Sennott’s approximating sequence method for showing that truncated MDPs converge to the countable state space problem. The ALP method can have much faster convergence, as demonstrated in numerical tests.